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Abstract 

This  project  has  tackled  the  problem  of  keeping  aligned  software  artifacts  and  non-functional  models  along  the 
software  lifecycle,  with  a  specific  focus  on  software  performance  aspects.  For  this  goal,  we  have  experimented  model- 
driven  techniques  that  allowed  to  introduce  automation  in  the  (forward  and  backward)  propagation  of  software 
model  changes.  This  document  reports  the  project  results. 
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For  sake  of  readability  of  the  project  results,  we  have  summarized  in  this  document  the  research  questions  addressed 
and  the  results  obtained,  whereas  for  details  about  methodologies  and  techniques  adopted  to  achieve  these  results  the 
readers  can  refer  to  the  10  papers  published  (plus  1  paper  under  revision)  thanks  to  this  project  funding  and  listed 
in  the  bibliography. 
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1.  Introduction 


This  3-years  project  has  focused  on  the  problem  of  filling  the  gap  between  non-functional  analysis  of  software 
models  and  software  development  artifacts.  Its  main  motivation  stems  from  the  fact  that,  still  today,  non-functional 
analysis  and  software  development  activities  are  quite  loosely  interconnected,  thus  the  additional  value  that  they 
can  lead  to  each  other  is  often  lost.  Instead  it  is  commonly  recognized  that  a  tighter  interconnection  of  practices, 
instruments  and  notations  would  give  rise  to  more  accurate  analyses  and  more  effective  feedback  from  analysis 
results  back  to  software  artifacts. 

The  confidence  that  advanced  model-driven  techniques  can  be  suitable  within  this  context  came  from  previously 
successful  collaborations  among  members  of  our  research  group  that  includes  researchers  from  the  Model-Driven 
Engineering  and  from  the  Non-functional  Analysis  areas. 

The  seed  of  this  project  activities  was  our  first  experiments  on  bidirectional  transformations  between  software 
artifacts  and  performance  models.  Our  intent  was  to  understand  whether  this  type  of  transformations  could 
close  the  loop  in  both  directions,  that  are:  transforming  software  artifacts  into  analyzable  performance  models 
(forward  path),  interpreting  performance  analysis  results  in  order  to  remove  problems  from  performance  models 
and  propagate  the  changes  back  to  software  artifacts  (backward  path)  [1], 

Starting  from  that  point,  the  project  has  addressed  three  main  research  questions  that,  along  the  project 
development,  have  emerged  as  key  points  in  the  backward  path  context,  that  is  the  most  problematic  one  [2].  They 
can  be  summarized  as  follows: 

RQ1  Is  it  more  advantageous  working  on  the  performance  side  or  on  the  software  side  to  solve  problems  identified 
with  performance  analysis?  Can  synergy  be  found  among  techniques  adopted  on  these  sides  (e.g.  bottleneck 
analysis  on  performance  side  and  antipattern  detection  and  removal  on  software  side)? 

RQ2  How  can  advanced  model-driven  techniques  help  in  the  model  refactoring  that  is  required  to  remove  perfor¬ 
mance  problems  detected  from  the  analysis? 

RQ3  How  performance  problem  detection  can  be  driven  towards  the  most  critical  problems  (i.e.  the  ones  that 
more  heavily  induce  bad  software  performance)? 

Several  results  have  been  obtained  for  all  the  above  research  questions.  The  remainder  of  this  report  is  organized 
as  follows:  the  next  three  sections  summarize  the  results  obtained  for  each  question,  then  conclusions  and  research 
perspectives  are  sketched. 

2.  Software  or  performance  side?  (RQ1) 

Several  approaches  have  been  introduced  in  the  last  few  years  to  tackle  the  problem  of  interpreting  model-based 
performance  analysis  results  and  translating  them  into  architectural  feedback.  Typically  the  interpretation  can  take 
place  by  browsing  either  the  software  model  or  the  performance  model.  In  [3]  we  have  compared  two  approaches 
that  we  have  introduced  for  this  goal:  one  based  on  the  detection  and  solution  of  performance  antipatterns,  and 
another  one  based  on  bidirectional  model  transformations  between  software  and  performance  models.  We  have 
applied  both  approaches  to  the  same  example  in  order  to  illustrate  the  differences  in  the  obtained  performance 
results.  Thereafter,  we  have  raised  the  level  of  abstraction  and  we  have  discussed  pros  and  cons  of  working  on  the 
software  side  and  on  the  performance  side. 

Then  we  have  investigated  whether  any  synergy  could  be  found  among  the  techniques  that  are  commonly 
adopted  on  the  two  sides,  i.e.  antipattern  detection  and  solution  on  the  software  side  and  bottleneck  analysis 
on  the  performance  side.  In  [4],  we  aimed  at  showing  that  the  approach  combination  allows  to  provide  software 
engineers  with  broader  sets  of  alternative  solutions  leading  to  better  performance  results.  We  have  explored  this 
research  direction  in  the  context  of  Layered  Queueing  Network  models.  After  comparing  the  results  achievable  with 
each  approach  separately,  we  have  quantitatively  shown  the  benefits  of  merging  bottleneck  analysis  and  performance 
antipatterns. 

3.  Model-driven  software  refactoring  (RQ2) 

The  contributions  brought  to  this  research  question  have  been  summarized  in  [5].  The  target  of  our  work  in 
this  direction  has  been  to  use  metamodeling  and  modeling  techniques  to  formalize  refactoring  actions  aimed  at 
removing  detected  performance  problems. 

In  [6],  we  have  introduced  an  approach  that  allows  the  refactoring  of  architectural  models,  based  on  antipatterns, 
aimed  at  providing  performance  improvement.  To  this  end,  we  used  a  Role-Based  Modeling  Language  to  represent: 
(i)  antipattern  problems  as  Source  Role  Models  (SRMs),  and  (ii)  antipattern  solutions  as  Target  Role  Models 
(TRMs).  Hence,  SRM-TRM  pairs  represent  new  instruments  in  the  hands  of  developers  to  achieve  architectural 
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model  refactorings  aimed  at  removing  sources  of  performance  problems.  Model  refactoring  for  antipattern  removal 
can  be  in  fact  obtained  by  replacing  an  SRM  with  the  corresponding  TRM.  This  approach  has  been  applied  to  a 
case  study,  whose  experimental  results  demonstrate  its  effectiveness. 

Thereafter,  in  [7]  we  have  represented  each  SRM-TRM  pair  as  a  difference  model  that  encodes  refactoring 
actions  to  be  operated  on  a  software  model  to  remove  the  corresponding  antipattern.  Differences  are  applied  to 
software  models  through  a  model  transformation  automatically  generated  by  a  higher-order  transformation. 

4.  Identifying  critical  performance  problems  (RQ3) 

The  effectiveness  of  performance  antipatterns  detection  is  affected,  like  any  pattern  detection  task,  from  the 
setting  accuracy  of  thresholds  that  reveal  the  antipattern  occurrence.  For  example,  an  antipattern  occurs  when  too 
many  messages  are  exchanged  between  two  software  components.  But  how  many  messages  are  too  many?  Hence, 
we  have  first  investigated  the  influence  of  thresholds  on  the  detection  task.  In  [8]  we  have  analyzed  how  a  set  of 
detected  antipatterns  may  change  while  varying  the  threshold  values.  This  work  has  been  then  consolidated  in 
[9],  where  we  have  also  studied  the  influence  of  thresholds  on  the  complexity  of  refactoring  actions,  and  we  have 
quantified  such  influence  using  precision  and  recall  metrics. 

With  reliable  detection  and  refactoring  techniques  in  the  hands,  the  next  problem  we  have  tackled  has  been 
the  introduction,  in  a  performance-driven  software  evolution  process,  of  metrics  that  can  drive  the  process  to 
convergence.  In  fact,  when  several  performance  antipatterns  are  detected,  a  critical  issue  is  to  decide  how  many 
and  which  ones  to  remove  first.  Starting  from  this  point,  we  have  developed  two  techniques:  (i)  one  based  on  a 
metric  of  guilt  that  helps  to  identify  the  antipatterns  that  more  heavily  contribute  to  the  violation  of  performance 
requirements  [10],  and  (ii)  another  one  aimed  at  working  in  a  non-deterministic  setting  of  thresholds,  with  the 
goal  of  identifying  the  antipatterns  that  more  likely  have  occurred  and  more  largely  can  improve  the  software 
performance  once  removed  [11]. 

The  former  technique  has  been  described  in  [10],  where  we  have  proposed  an  approach  that  provides  software 
performance  feedback  to  software  engineers,  since  it  suggests  the  design  alternatives  that  allow  overcoming  the 
detected  performance  problems.  The  feedback  process  may  be  quite  complex  since  engineers  may  have  to  assess 
several  design  options  before  achieving  the  architectural  model  that  best  fits  the  end-user  expectations.  In  order 
to  optimise  such  process  we  have  introduced  a  ranking  methodology  that  identifies,  among  a  set  of  detected 
antipatterns,  the  guilty  ones,  i.e.  the  antipatterns  that  more  likely  contribute  to  the  violation  of  specific  performance 
requirements.  The  introduction  of  our  ranking  process  leads  the  system  to  converge  towards  the  desired  performance 
improvement  by  discarding  a  consistent  part  of  design  alternatives. 

The  latter  technique  is  described  in  [11],  where  we  have  worked  in  a  fuzzy  context  with  threshold  values  that 
cannot  be  determined,  but  only  their  lower  and  upper  bounds  do.  On  this  basis,  the  detection  task  produces  a  list 
of  performance  antipatterns  along  with  their  probabilities  to  occur  in  the  model.  Several  refactoring  alternatives 
can  be  available  to  remove  each  performance  antipattern.  Our  approach  associates  an  estimate  of  how  effective  each 
alternative  can  be  in  terms  of  performance  benefits.  We  have  demonstrated  that  the  joint  analysis  of  antipattern 
probability  and  refactoring  benefits  drives  the  designers  to  identify  the  alternatives  that  heavily  improve  the  software 
performance. 

5.  Conclusions 

This  3-years  project  has  allowed  us  to  investigate  multiple  aspects  of  software  evloution  driven  by  performance 
aspects.  As  summarized  in  this  report,  the  results  of  this  project  have  been  published  in  several  conference  and 
journal  papers. 

This  work  can  be  developed  in  future  in  several  directions,  where  the  following  ones  seem  to  us  very  promising. 

First  of  all,  it  would  be  very  interesting  to  experiment  the  techniques  developed  here  on  real-world  examples 
of  performance-critical  systems.  This  would  allow  to  consolidate  the  validation  of  the  whole  process.  At  the  same 
time,  the  application  of  such  techniques  on  specific  application  domains  (such  as  Wireless  Sensor  Networks,  Cloud, 
Safety-Crictical  Systems,  Adaptive  Software  Systems)  would  help  to  embed  the  process  in  specific  context  where 
perhaps  some  steps  have  to  be  refined  in  order  to  be  more  effective.  For  example,  we  have  recently  started  to  study 
the  application  of  Feedback  Control  Theory  to  Adaptive  Queueing  Networks,  that  are  performance  models  where 
several  aspects  can  change  in  order  to  maintain  a  certain  non-functional  objective. 

In  another  direction,  the  extension  of  this  project  results  to  other  non-functional  aspects  of  software  would  also 
be  very  interesting  to  investigate.  Beside  the  canonical  -ilities,  such  as  reliability  and  availability,  techniques  that 
analyze  tradeoff  among  these  properties  in  order  to  generate  feedback  to  software  engineers  are  still  lacking.  This 
problem  is  ever  more  critical  today,  due  to  the  increasing  heterogeneity  of  environments  where  software  is  deployed, 
and  also  due  to  the  adverse  conditions  under  which  software-based  devices  are  required  to  be  used  (e.g.  thermal 
conditions  of  sensors  in  different  environments  like  ships,  wind  devices,  data  centers,  etc.). 
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